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Enrolling in college and earning a degree or certificate is 
one of the primary pathways to economic success; how- 
ever, many students who begin a college education fail to 
complete it. In 2017, median weekly earnings of full-time 
workers with an associate degree was 17 percent higher than 
full-time workers with a high school diploma only.' Despite 
the economic advantages of educational attainment, only 32 
percent of students enrolled in two-year institutions com- 
plete their programs within three years, while 60 percent 
of students enrolled in four-year institutions complete their 
programs within six years. The first year of college is critical, 
as 38 percent and 19 percent of first-time, full-time students 
in two-year and four-year institutions, respectively, fail to 
return to the same institution for their second year.” As a 
result, several interventions have been designed to support 
student retention and degree completion among college 
students. One example is Success Boston Coaching. 


Success Boston Coaching is a coaching intervention for stu- 
dents who are traditionally underrepresented in college to 
help them transition from high school to college and prog- 
ress in college. Students are paired with a dedicated coach 


starting as early as the spring of their senior year of high 
school and receive coaching through their first two years in 
college. As Boston’s citywide college completion initiative, 
Success Boston partners with existing nonprofit organiza- 
tions focused on coaching and mentoring to deliver these 
one-on-one coaching services. Nonprofit coaching partners 
may also provide students with other direct services such as 
tutoring and career readiness support, and financial support 
that includes scholarships, transportation subsidies, and 
funding for school-related materials and supplies. 


This What Works Clearinghouse (WWC) report, part of 

the WWC’s Supporting Postsecondary Success topic area, 
explores the effects of Success Boston Coaching on progress- 
ing in college and academic achievement. The WWC identi- 
fied seven studies of Success Boston Coaching. Two of these 
studies meet WWC standards.* The evidence presented in 
this report is from studies of the impact of Success Boston 
Coaching on high school and postsecondary students— 
including Asian, African-American, Hispanic, and White 
students—in a variety of school settings, including urban and 
suburban areas. 


What Happens When Students Participate in Success Boston Coaching? 


The evidence indicates that implementing Success Boston 
Coaching: 


¢ may increase progression in college 
¢ may increase academic achievement 


Findings on Success Boston Coaching from two studies 
that meet WWC standards are shown in Table 1. The table 


reports an effectiveness rating, the improvement index, and 
the number of studies that contributed to the findings. The 
improvement index is a measure of the intervention’s effect 
on an outcome. It can be interpreted as the expected change 
in percentile rank for an average comparison group student 
if that student had received the intervention. 


Table 1. Summary of findings on Success Boston Coaching from studies that meet WWC Standards 


Study findings Evidence meeting WWC standards (version 4.0) 
Improvement index Number Number 
Outcome domain Effectiveness rating (percentile points) of studies of students 
Progressing in college Potentially positive effects +9 2 8,375 
Academic achievement Potentially positive effects +4 2 5,543 


Note: The improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the 
intervention. For example, an improvement index of +4 means that the expected percentile rank of the average comparison group student would increase by 4 points if the 
student received Success Boston Coaching. The improvement index values are generated by averaging findings from the outcome analyses that meet WWC standards, as 
reported by Linkow et al. (2017) and Linkow et al. (2019). A positive improvement index does not necessarily mean the estimated effect is statistically significant. Progressing 
in college outcomes reported in these studies include college persistence to the second year of higher education. Academic achievement outcomes reported in these studies 
include cumulative grade point average. The effects of Success Boston Coaching are not known for other outcomes within the Supporting Postsecondary Success topic 
area, including college enrollment, college attendance, postsecondary degree attainment, credential attainment, employment, and earnings. 


BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 4.0) and the Supporting Postsecondary Success topic area protocol (version 4.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and the 
number of studies that tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported 
results and about what will happen if the same intervention is implemented again. The following key explains the relationship between 


effectiveness ratings and the statements used in this report: 


Effectiveness rating Rating interpretation 


Description of the evidence 


Positive (or negative) effects The intervention is likely to change an Strong evidence of a positive effect, with no 


outcome 


overriding contrary evidence 


Potentially positive (or negative) effects The intervention may change an outcome Evidence of a positive effect with no overriding 


contrary evidence 


No discernible effects The intervention may result in little to no No affirmative evidence of effects 


change in an outcome 


Mixed effects The intervention has inconsistent effects Evidence includes studies in at least two of 


onan outcome 


How is Success Boston Coaching Implemented? 


The following section provides details of how Success Boston 
Coaching was implemented. This information can help edu- 
cators identify the requirements for implementing Success 
Boston Coaching and determine whether implementing this 
intervention would be feasible in their districts or schools. 
Information on Success Boston Coaching presented in this 
section comes from the studies that meet WWC standards 
(Linkow et al., 2017; Linkow et al., 2019), Success Boston’s 
website, and from correspondence with the developer. 


e¢ Goal: Success Boston Coaching aims to improve the 
college completion rates of Boston-area high school 
graduates by providing one-on-one transition coaching 
to students starting as early as their senior year in high 
school through their first two years in college. 


e Target population: Success Boston Coaching is designed 
for high school seniors in transition to attending two- 
and four-year colleges in the Greater Boston area who 
are traditionally underrepresented in college, such as 
low-income students, first-generation students, and 
students of color. Students are eligible to participate in 
Success Boston Coaching as early as the spring of their 
senior year of high school. 


e Method of delivery: Success Boston Coaching provides 
one-on-one transition coaching through nonprofit 
coaching partners that have included the Boston 
Private Industry Council, Bottom Line, College Bound 
Dorchester, Freedom House, Hyde Square Task Force, 


these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


Sociedad Latina, The Steppingstone Foundation, West 
End House Boys & Girls Club, and uAspire. These 
partners range from national college access and success 
organizations to local community-based organizations 
that serve populations traditionally underrepresented 
in college. Through in-person, phone, email, social 
media, and text communications, students receive 
sustained and proactive support from coaches staffed 
by the partner organizations. 


Frequency and duration of service: Students can 
receive coaching as early as the spring of their senior 
year of high school and coaching can continue through 
their first two years in college. Students’ one-on-one 
interactions with their coaches may range from three to 
15 times per year and last between 25 and 40 minutes. 


Intervention components: Success Boston Coaching 
is comprised of the following key components: one-on- 
one transition coaching, financial aid information and 
nudges, other direct services, financial support, and 
professional development for coaches. Refer to Table 2 
for additional details. 


Comparison group: In the two studies that contribute 
to this intervention report, students in the comparison 


group received traditional counseling supports 
provided by their high school and college. 


N 


Table 2. Components of Success Boston Coaching 


Key component Description 

One-on-one Students receive one-on-one transition coaching starting as early as the spring of their senior year in high school and coaching can 

transition continue through their first two years in college. Coaching is primarily conducted in-person, but students may reach their coaches 

coaching via phone, email, text message, or social media. Coaches meet with students as frequently as needed throughout their 
participation. As of the 2019-20 academic year, students meet with their coaches at least two times per month. Topics addressed 
during coaching sessions include academics, financial aid, career planning, and managing life responsibilities. Depending on the 
nonprofit coaching partner, coach caseloads range from 12 to 65 students; in the 2015-16 academic year, students’ one-on-one 
interactions with their coaches ranged from three to 15 per year and lasted between 25 and 40 minutes. 

Financial aid Success Boston Coaching partners with uAspire, a national nonprofit organization focused on increasing students’ knowledge of 


information and 
nudges 


Other direct 
services 


Financial 
support 


Professional 
development 
for coaches 


financial information and resources available to help make college affordable for students. uAspire supports students in completing 
financial aid forms. Additionally, uAspire administers a text messaging program for students during their first year of college to remind 
them of important financial aid deadlines and provide financial aid-related information and resources. Students can text back to the 
messages to receive help from uAspire staff. Students receive, on average, eight messages during their first year of college through 
the following summer. 


Coaching partners may offer tutoring for students. They may also host special events for students, such as career night presentations 
or mock job interviews with volunteer professionals from the community. 


Coaching partners may offer scholarships, transportation subsidies, or financial support for school-related materials, such as supplies. 


Coaching partners provide their coaches with training. The content and duration of the trainings vary based on organization. Coaches 
participate in Success Boston Coaching network-wide monthly meetings for continuing professional development and coaches can 
request specific training topics for these meetings. Example topics include: financial aid and financial literacy, coaching styles, building 
relationships with students, and bolstering student engagement. In addition, uAspire provides professional development for coaches 
on financial aid topics and processes. Some colleges hold ongoing monthly or biweekly meetings for all coaches on their campuses. 


What Does Success Boston Coaching Cost? 


The WWC identified a number of cost components from 2015. For the 2015-16 academic year-the first year in which 
the most recent study reviewed (Linkow et al., 2019) and the intervention was scaled from serving about 300 students 
the Success Boston website. This preliminary list of costs is to 1,000 students per cohort annually—the estimated total 
not designed to be exhaustive; rather, it provides educators cost was $5,762 per student, which accounts for both the 

an overview of the major resources needed to implement scale-up expenses and the administrative costs to fulfill 
Success Boston Coaching. The intervention costs described in Social Innovation Fund (SIF) grant reporting requirements. 
Table 3 are based on the information available as of Without the additional administrative costs associated with 
March 2020. program scale-up, in the 2016-17 academic year, the cost per 


Success Boston Coaching was awarded a Social Innovation 
Fund (SIF) grant to scale up coaching services in the fall of 


student decreased to $3,037. Costs may vary depending on 
the size and type of organization delivering the coaching. 


Table 3. Cost ingredients for Success Boston Coaching 


Costingredients Description Source of funding 

Personnel Success Boston Coaching is staffed by coaches from nonprofit coaching organizations who The Boston Foundation (TBF) 
provide direct services to students; program administrators to support the program and with support from a Social 
conduct recruitment activities; and college liaisons to coordinate campus coaching activities. Innovation Fund (SIF) grant from 
Success Boston Coaching also contracts with uAspire to provide staff who deliver financial the Corporation for National and 
aid training for coaches, FAFSA completion events for students, and a text messaging Community Service (CNCS); 
program for students focused on sharing financial aid-related information and resources. coaching partner organizations 

Facilities Facilities costs include space for administrative staff and for providing services. Success Coaching partner organizations; 
Boston Coaching relies on partner organizations to provide space for service delivery. colleges 
Some partners rely on dedicated campus space such as cubicles, scheduled time in offices, 
or classrooms provided by the colleges. 

Equipment and — Standard information technology (IT) equipment such as computers, phone, printers, and Coaching partner organizations; 

materials photocopiers for staff is required. colleges 

Direct Students may receive some direct cash or cash-equivalent assistance such as reduced-fare Coaching partner organizations; 

assistance public transportation passes, scholarships, and school supplies. transit authority 

Other Depending on the partner organizations, students may also receive additional supports such Coaching partner organizations; 


as tutoring, career night presentations or mock job interviews, and annual events. volunteers; AmeriCorps members 


Note: The Boston Foundation (TBF) provides approximately $1 million per year to fund coaching and other activities for transition coaching, such as training and supports 
for coaches and program management. Between the fall of 2015 and the fall of 2019, Success Boston received funding from a Social Innovation Fund (SIF) grant from the 
Corporation for National and Community Service (CNCS). The SIF grant required a match from The Boston Foundation and a local match from the nonprofit coaching partners. 


For More Information: 


About Success Boston Coaching 
The Boston Foundation 
75 Arlington Street, 3rd Floor 
Boston, MA 02116 
Attn: Antoniya Marinova, Assistant Director, Education to Career 
Email: Antoniya.Marinova@tbf.org Web: www.successboston.org Phone: (617) 338-3928 
About the cost of the intervention 
Web: www.successboston.org 
About the studies reviewed 
Linkow, T., Gamse, B., Unlu, F., Bumgarner, E., Didriksen, H., Furey, J., Meneses, M., Sami, M., & Nichols, A. (2017). The power 
of coaching: Interim report on the impact of Success Boston’s transition coaching on college success. Cambridge, MA: Abt 
Associates. Retrieved from https://www.tbf. org/-/media/tbforg/files/reports/sb-31 7-interim-outcomes-report-final.pdf 
Linkow, T., Bumgarner, E., Didriksen, H., Lack, K., Nichols, A., Dastrup, E., Dastrup, S., & Gamse, B. (2019). The story of scaling up: 
Interim report on the impact of Success Boston’s Coaching for Completion. Cambridge, MA: Abt Associates. Retrieved 
from https://eric.ed.gov/?id=ED602748 


In What Context Was Success Boston Coaching Studied? 


The following section provides information on the setting This information can help educators understand the context 
of the two studies of Success Boston Coaching that meet in which the studies of Success Boston Coaching were con- 
WWC standards, and a description of the participants in the ducted, and determine whether the intervention might be 
research. suitable for their setting. 


WHERE THE STUDY WAS CONDUCTED 
2 studies, 8,375 students in Massachusetts 
Districts: Urban and suburban school districts 


African Asian White Unspecified Female Male 
American 


Free & reduced- Special English ; 
price lunch: education: learners: Grades: iD —-¢@ @® © 


77% 11% 11% 12-Postsecondary 


LEARN MORE 


Read more about the Success Boston Coaching intervention and the studies that are summarized here in 
LW the Intervention Report. 
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